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AN ANALYSIS OF A FIRST-ORDER POLYNOMIAL 
PREDICTOR FOR DATA COMPRESSION^ 

By M. Melvin Bruce 
Langley Research Center 

SUMMARY 

A theoretical analysis is presented of a first-order polynomial predictor for use in 
data compression. The investigation is concerned with compressing data which existed 
in the form of samples taken at a constant rate. This analysis is also limited to data 
having a Gaussian distribution of amplitude. Four pseudo-random time histories with 
different amplitude spectra are compared by using this method. A digital computer was 
used to generate data approximating the four time histories and to measure the compres- 
sion possible in the generated data for comparison with the theoretical calculations. 

The results of this investigation show that, under known conditions, it is possible to 
calculate the amount of data compression which can be obtained through the use of a 
first-order polynomial predictor. This investigation also shows that high frequencies in 
the data reduce the amount of compression and that there is a trade-off between the 
amount of error introduced by the compression system and the amount of compression 
obtained. This method of analysis is limited to data with known statistics. 

INTRODUCTION 

Compression of sampled data is defined as the reduction in the total number of 
samples required for the reconstruction of the original data within a given accuracy. 

One of the techniques for compressing data is the application of polynomial prediction. 
The amplitude of a given sample is predicted as a function of the previous samples. The 
order of the prediction scheme indicates the order of the polynomial equation and the 
number of previous samples used for prediction. A zero-order polynomial predictor 
uses only one previous sample and predicts that the succeeding sample will have the 
same amplitude as the previous sample. A first-order polynomial predictor uses two 
previous samples to generate amplitude and slope information for its prediction. 

^The information presented herein was offered as a thesis in partial fulfillment of 
the requirements for the degree of Master of Electrical Engineering, University of 
Virginia, Charlottesville, Virginia, August 1965. 


Among those actively conducting research in this area are Gardenhire (ref. 1) and 
Medlin (ref. 2). Their work has been essentially experimental and has consisted in 
making compression measurements on data from actual telemetry systems or on arti- 
ficially generated data. There has been a scarcity of work directed toward calculating 
the amount of compression possible by polynomial prediction from the statistics of the 
data. Hochman in reference 3 analyzed theoretically the compression by a zero-order 
polynomial predictor. The purpose of the present investigation is to analyze theoreti- 
cally the first-order polynomial predictor for data compression, using Hockman's tech- 
niques as an outline, and to compare these results with actual measurements on data 
simulated by a computer. The first-order polynomial predictor was analyzed by using 
assumed statistics for the data and by solving a random walk problem. 

The results of this investigation show that the amount of data compression for a 
first-order polynomial predictor can be calculated by using the statistics of the data. 
This method should be a useful tool to a designer contemplating the use of first-order 
polynomial prediction for data compression. 

SYMBOLS 


Cjj compression ratio 

E(x) expected value 

F(co) amplitude spectrum 

fjjj maximum frequency component 

fg sampling frequency 

K tolerance limit as a function of a 

kj amplitude of frequency component 

Lii,Li2,’L2i,L 22» • • plus and minus tolerance limits 

||m|| matrix 

|m| determinant of matrix j|Mj| 

Mjj cofactor of determinant |m| 

n number of samples since a change in the reference samples 

p(x) probability density function 
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t 


time 


U difference between A's 

Xj^ amplitude of nth sample 

Zjj difference between actual amplitude and predicted amplitude 

A difference in amplitude between adjacent samples 

p correlation coefficient 

a standard deviation 

a2 variance 

T time displacement 

i//(t) moment generating function 

o) angular rate in radians per second 

maximum angular rate 

THEORETICAL ANALYSIS OF A FIRST-ORDER POLYNOMIAL PREDICTOR 

In general, a first-order polynomial predictor performs a prediction of the ampli- 
tude of a given sample by using information on both amplitude and slope that has been 
obtained from previous samples. The particular first-order predictor to be considered 
herein uses two adjacent samples to predict the amplitude of succeeding samples. The 
general equation for this type of prediction is: 

Xn = xj + (n - 1)(X2 - 

In this analysis the samples are assumed to be evenly spaced in time. 

Figure 1 shows the operation of the first-order polynomial predictor in data com- 
pression. The curve in figure 1 is an amplitude -time plot of the input data. The data 
have been sampled at a rate fg and the samples have an amplitude Xn- Two adjacent 
samples, xj and X£, are chosen as the first reference samples. A reference line is 
drawn between the two points; predicted amplitudes of future samples will fall along this 
line. Allowable tolerance limits, Lji and L12, are placed on either side of the ref- 
erence line. All succeeding samples are discarded imtil a sample falls outside the tol- 
erance band. In figure 1 the first sample to fall outside the tolerance band is the fifth 
sample. When this phenomenon occurs, the previous sample, X4, is used in conjunction 
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with Xg to generate a new reference line. The previous sample is used so that the data 
may be reconstructed by joined straight line segments. The maximum error in the 
reconstructed data due to the compression is one -half the width of the tolerance band. 

In order to analyze the first-order predictor theoretically, the following assump- 
tions are made: 

(1) The power density spectrum of the data is known. 

(2) The probability density of the amplitude of the data is Gaussian with a mean of 
zero and a standard deviation of a. 

(3) The sampling rate is greater than twice the highest frequency present in the 

data. 

As assumption (2) shows, this analysis is concerned only with the Gaussian distribution 
of amplitudes. However, the method covild be applied td other probability distributions. 

The compression ratio is defined as the total number of samples handled divided 
by the number of samples retained. The retained samples are the two adjacent samples 
which are used to generate the reference line. The method of analysis is similar to that 
of Hochman (ref. 3). The problem is approached by solving a random walk problem in 
order to find the average number of samples required to reach one of the tolerance 
limits. This number is the average number of samples handled, and the number of sam- 
ples retained is two. Therefore, the theoretical compression ratio is one -half the aver- 
age number of steps to the boundary. 

The difference in amplitude between the actual amplitude of the nth sample and the 
predicted value of the nth sample is Zj^. In appendix A, Zn is found to be 

n 

Zn= ^ Ui 

i=l 


where 


Ui = (xi+2 - Xi+i) - (X2 - xj) 

Ui ^ Ai+1 - 

and is the difference in amplitude between samples xj^2 ^1+1 • 

The probability density for Un can be found by using a joint probability density 
function for four variables, p(xj,X2,Xjj^j^,Xjj^2)* ^ 

X2 - Ai = xj 
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Un - ^n+1 " ^1 

pOO 

P(Un) = J ^ P(^n+1 “ Un, An+l) d 


O Discarded siijiples 
□ Retained samples 



Time 


Figure L- A first-order polynomial predictor. 
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The probability density function for Un is derived in appendix B and is found to be a 
Gaussian density function with mean equal to zero. 

In reference 5, Blackwell and Girshick solve a problem in sequential analysis, 
which is in essence a random -walk problem. Appendix C presents this solution, in the 
notation used herein, to obtain the expected value of Zn^: 

eM = L11L12 = KiK 2 a 2 ( 1 ) 

where 0 ^ is the variance of the amplitude of the data and 


Lii = K^a 


Li2 = K2<y 

the quantities and Lj2 the plus and minus tolerance limits. By the definition 

of Zji, together with the definition for the second moment (ref. 6), 

n n n 

i=i i=i 3=1 

and where i is never equal to j. In this analysis 
n n 

I y E(UiUj) = 0 (l*i) 

i=l j=l 

This assumption is considered reasonable since the expected value of UiUj may be 
either positive or negative. This relationship, in effect, assumes that there is no corre- 
lation between Uj and Uj for i # j. 

If the plus and minus tolerance limits are assumed to be equal and equations (1) 
and (2) for e(Zji^) are equated. 


e{zJ) = 

where is the variance of Uj 

it is assumed that 
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K = Ki = K2 


k2(t 2 = ^ au^ (3) 

i=l ^ 

The n in equation (3) is the average number of steps to one of the tolerance limits. It 
has already been determined that the compression ratio Cjj is 


n - 

^R'2 


However, it is not possible to solve for the compression ratio explicitly as a function of 
K, but it is possible to solve for K as a function of Cj^. 


K = 


i 

a 




The values of 
values of i. 


were calculated from the equation 
These values were then used to compute 


obtained in appendix B for various 
K as a function of Cj^. 


Figure 2 is a plot of the theoretical compression ratio for a first-order predictor 
operating on four types of input data. In this figure the sampling rate was 5 times the 
cutoff frequency of the data. The abscissa is K, the allowable tolerance limit expressed 
in terms of a, which is the standard deviation of the amplitude of the data. 


The amplitude spectra for the foiir types of input shown in figure 2 are: 


L spectrum: 


F(w) = ki 

(For 0 ^ la>l ^ O.lcojn^ 

F(o)) = O.Olki 

(For O.lojm ^ loj| S tOm) 

F(co) = 0 

(For other values of u>) 
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Exponential spectrum: 


Triangular spectrum: 


F(o.) = kj exp^- 
F(o>) = 0 


(For 0 £ iw| ^ cOm) 
(For other values of a>) 


F( 0 )) = kg^l - (For 0 £ |w| ^ Wm) 

F(w) = 0 (For other values of w) 


Rectangular spectrum: 


F(w) = k4 


(For 0 i |o)| ^ Wm) 


F(w) = 0 


(For other values of w) 


Figures 3, 4, 5, and 6 show the amplitude spectra and the correlation coefficients 
for the four types of input. The abscissa for the correlation coefficient is the displace- 
ment n expressed as a multiple of the number of samples away from the first refer- 
ence sample with the sampling rate being five times the maximum frequency in the data. 
For example, the correlation between adjacent samples, taken at the given sampling 
rate, is found from the curves at a displacement equal to one. 
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Figure 2.- Theoretical compression ratio for a first-order predictor. 
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Correlation coefficient ^ , Correlation coefficient 



EXPERIMENTAL PROCEDURE 

Since all experimental work involved sampled data, a breadboard model of the 
first-order predictor for data compression would have been a special purpose digital 
computer. For this reason no breadboard model of the predictor was constructed, but 
all experimental work was performed on a digital computer located at the Langley 
Research Center. 

Data were generated by computer approximations of the characteristics used in 
the theoretical analysis. These data were then examined by the computer to determine 
the amount of compression obtainable by a first-order predictor as a function of the 
width of the tolerance band K. 

The procedure used for generating the data with the desired characteristics was 
as follows; 

(1) Pseudo-random numbers were generated by approximating a Gaussian proba- 
bility density with mean of zero, variance of one (refs. 7 and 8), and a flat amplitude 
spectrum. 

(2) Numerical filtering (ref. 9) by use of weighting functions and a convolution 
transformed the flat amplitude spectrum into that of the four spectra used in the theo- 
retical analysis. 

These two steps gave a sequence of numbers for each of the four spectra. If this 
sequence of numbers is taken to be a sequence of samples of a continuous fxmction, the 
continuous function would have a probability density approximating a Gaussian probabil- 
ity density function and amplitude spectra approximating that of one of the four spectra. 

The procedure used for measuring the compression possible on the data generated 
by a first-order predictor was as follows: 

(1) The first two samples, xj and X2, of the sequence were used to find the slope 
of the reference line, Aj; 


Ai = X2 - xi 


(2) The predicted value of X3 is X2 + Aj. 

(3) The difference in the actual value of X3 and the predicted value was calculated. 
If this difference was less than the width of the tolerance band, the sample was considered 
redundant. The amplitude of X4 was then predicted by X2 + 2Aj. This process was 
continued until the difference in actual and predicted value of a sample was greater than 
the width of the tolerance band. 
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(4) When a nonredundant sample was located, the computer returned to step (1) 
with the nonredimdant sample as X£ and the previous sample as xj. 

(5) A record was kept of the number of samples examined and the number found to 
be nonredundant. The compression ratio is the total number of samples divided by twice 
the number of retained samples. 


RESULTS 

The simulated data generated by the computer was a sequence of 5000 numbers. 
The autocorrelation of the pseudo-random numbers was calculated to check the random- 
ness of the numbers. At all nonzero displacements, the autocorrelation was found to be 
less than 3 percent of the value at zero displacement. Figure 7 is a plot of the first one 
hundred points of the pseudo-random data before being filtered. Figures 8, 9, 10, and 11 
show the data after being filtered to conform to the spectral requirements of the four 
different inputs. These figures, as expected, show that the data having the rectangular 
spectrum has the greatest content of high frequencies. 



Figure 1 - Random data. Gaussian density before filtering. Mean, 0.000849; 
standard deviation, 1.003534. 


13 



Amplitude Amplitude 


2 



Time 

Figure 8.- Random data. Gaussian density with L-spectrum filter. Mean, 0.004785; 
standard deviation, 0.943357. 



Time 

Figure 9.- Random data. Gaussian density with exponential-spectrum filter. Mean, 0.003667; 

standard deviation, 0.966783. 
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Amplitude 



Figure 1 



The amplitude spectrum was measxired from the actual data in order to make a 
comparison with the theoretical spectra. Figures 12, 13, 14, and 15 show both theo- 
retical and measured amplitude spectra for the four inputs. It can be observed from 
these figures that the actual amplitude spectra have the general shape of the theoretical 
spectra. The difference in the two curves is due to the fact that numerical integration 
was used in filtering the data with only forty -one points used to approximate the ampli- 
tude spectra. 



Figure 12.- Amplitude spectrum for L-spectrum filter. 



Figure 13.- Amplitude spectrum for exponential-spectrum filter. 
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© 

Frequency 

Figure 15.- Amplitude spectrum for rectangular-spectrum niter. 



The data were also checked to insure 
that the amplitude distribution followed a 
Gaussian probability curve. Figure 16 is a 
plot of the cumulative relative frequency of 
the random data before filtering, which has 
been plotted on Gaussian probability paper. 

If the data do have a Gaussian distribu- 
tion, the points will form a straight line. 

The random data before and after filtering 
were found to approximate Gaussian distrib- 
uted data out to approximately ±2. 5a. Fig- 
ures 17, 18, 19, and 20 are the plots of the 
cumulative frequency for the data after it 
had been filtered according to the require- 
ments of the four spectra. 

Figure 21 shows the measured com- 
pression ratio as a function of the width of 
the tolerance band for the first-order pre- 
dictor. The measured values of the com- 
pression do not agree exactly with the theo- 
retical values. However, the theoretical 
and actual compression ratios have the 
same shape, general range of values, and response to the four types of input data. Both 
the theoretical and measured curves show that the presence of high frequencies in the 
data reduces the amount of data compression which can be obtained. It is anticipated 
that the differences between the measured and theoretical compression ratios might be 
reduced by using data which more closely approximate the theoretical data. However, 
the results obtained herein are sufficient for comparison with theoretical results. 

It can be seen from the curves in figures 2 and 21 that there is a tradeoff between 
the amount of compression and the width of the tolerance band. As the compression 
system is allowed to introduce more error with sample rate held constant, the amount 
of compression is increased. 



Sample amplitude 

Figure 16.- Random data before filtering. 
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0 .1 .2 .3 .4 .5 .6 .7 

Tolerance limits, K 

Figure 21.- Actual compression ratio. 

CONCLUDING REMARKS 

The results of this investigation have shown that it is possible under some condi- 
tions to calculate the amount of data compression which can be obtained by using a first- 
order polynomial predictor. This method is limited to the cases in which the statistics 
of the data are known; that is, the amplitude spectrum and the probability distribution of 
amplitudes are known. This investigation has been concerned only with data having a 
Gaussian probability distribution simply for mathematical convenience but could be 
extended to other probability distributions. This investigation has also shown that there 
is a tradeoff between the amount of error introduced by the compression system and the 
amount of compression obtained. 

The methods of analysis presented should be useful as a basis for the extension of 
theoretical analysis to other types of first-order predictors and to data with other types 
of probability distributions. 

Langley Research Center, 

National Aeronautics and Space Administration, 

Langley Station, Hampton, Va., October 27, 1966, 

125-21-02-04-23. 
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APPENDIX A 


DERIVATION OF THE EQUATION FOR THE DISTANCE FROM THE 
PREDICTED VALUE IN A FIRST-ORDER SYSTEM 

Definitions of quantities required for derivation of an equation for the distance 
from the predicted value in a first-order system are as follows: 

Xj amplitude of the ith sample 

Aj difference in amplitude of the (i + 1) sample and the ith sample; that is, 

M = Xi+1 - Xi 

reference difference; Ax = X 2 - Xj 

Uj difference in amplitude of the (i + 1) difference and the reference difference; 

Ui = - Aj 

n number of samples beyond the second reference sample; that is, n = i - 2 

Zjj difference in amplitude between the actual sample and the predicted value of 

the nth sample; that is, = Xj^_^2 " [^l + (n + l)Ai] 

From these definitions, 

Zn = Xn+2 - |xi + (n + 1 )a{] 

Zn = Xn+2 - xj - (n + l)Ai 

Zn = (xn+2 - Xn+l) + (xn+1 - Xn) + . . . + (x 2 - xj) - (n - l)Ai 

Zn = An+1 + An + An_i + . . . + Ai - (n + l)Ai 

Zn = (An+1 - Ai) + (An - Ai) + . . . + (A 2 - Ai) 

Zn = Un + Un-1 + • . . + 
n 

z„ = 2 Uj 

j=l 
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APPENDIX B 


DERIVATION OF THE PROBABILITY DENSITY OF Un 

If xj, X 2 , Xji+i, and x^+2 are variables with Gaussian probability densities 
and each has a mean of zero and a standard deviation of a, 


P(Xi) 






2cr2y 


and the joint probability density function for four variables is (ref. 10) 


p(xi,X 2 ,Xn+i,Xn+ 2 ) ;f= exp 

{.2n)^\[\M\ 


S 1 “‘pi’'! 


2|m1 . 

i,]=l 


In the preceding equation, |m| is the determinant of the matrix ||m||. 


in which 


||M|| = Matrix 


dll 

dl2 

dl3 

di4 

d21 

d22 

d23 

d24 

dsi 

dS2 

d33 

CO 

d4i 

d42 

d43 

d44| 



and the M^j's are the cofactors of the determinant. The coefficient p(l) is the cor- 
relation coefficient between samples which are separated by one sample time (xj and X 2 , 
^n+1 ^n+2)j ~ 1) correlation coefficient between samples which are 

separated by (n - 1) sample times (X 2 and x^+j); p(n) is the correlation coefficient 
between samples which are separated by n sample times (xj and x^+i, X 2 and 
Xn+2)j + 1) the correlation coefficient between samples which are separated 

by (n + 1) sample times (x^ and Xjj^2)* Therefore, 


22 



APPENDIX B 


|m| = a®|j3(l)2 - p(n)2] + |J)(n - l)^ - ^[p(n + 1)^ - l] 

- 2[p(l)^ +p(n)^|l +p(n - l)p(n + + 4p(l)p(n)[p(n - 1) +p(n + 1) 


Mil = M44 = a® jl + 2p(l)p(n - l)p(n) - p(l)2 - p(n - 1)^ - p(n)^ 

Mi 2 = M21 = M34 = M43 = o6jp(n)p(n + 1) + p(l)^ +p(n - l)p(n) - p(l) - p(l)p(n)2 

- P(l)p(n - l)p(n + 1 ^ 

Mi 3 = M31 = M24 = M42 = o6jp(l)p(n - 1) + p(l)p(n + 1) + p(n)^ - p(n - l)p(n)p(n + 1) 

- p(l)2p(n) - p(n)J 

Mi 4 = M41 = CT6|p(n - l)^p(n + 1) + 2p(l)p(n) - p(l)^p(n - 1) - p(n + 1) - p(n - l)p(n)^ 

M23 = M32 = o6jp(n - l)p(n + 1)2 + 2p(l)p(n) - p(n - 1) - p(n)^p(n + 1) - p(l)^p(n + 1^ 

M22 = M33 = o6|l + 2p(l)p(n)p(n + 1) - p(n + 1)2 - p(l)2 _ p(n)^ 

p(Xi,X2,Xn+l,x„ 2) = K 

^ ( 27 t)^ 

+ (Mi2 + M2i)xiX2 + (Mi 3 + M 3 i)xiXn+i + (M14 + M 4 i)xiXn +2 

+ (M23 + M32)x2Xn+i + (M24 + M42)x2Xn+2 + (M34 + M43)xn+lXn+^ 


M22X22 + M33X„2^ + M44Xn^2 
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APPENDIX B 


Let xj = X2 - Aj. Then 


^OO 

P(^LXn+i,Xn+2) = J ^ p(x2 - Al,X2,Xn+i,Xn+2)dx2 


Let Xn+i = Xn +2 “ An+i- Then 


pOO 

p(Ai,An+ij = j ^ P(Al,Xn+2 - An+i,Xn+2)dxn+2 


P ( Al , An + i ) = 


l/N 


exp< 


-1 


2 

- HiH 4 ^ + H2H3H4J 


| m |( 4 Hi 2 - H32 ) 


| h 5 ( 4 Hi 2 - H32 ) - H 22 hi 


where 


Hi = Mil + 2 Mi 2 + M22 


H2 = 2AijMii + M12J + 2An+ijMi3 + M2^ 

H3 = 2|2Mi3 + Mi4 + M2^ 

H4 = 2AijMi3 + Mi^ + 2 An+l ^33 + ^3^ 

H5 = MiiAi 2 + 2 Mi 3 AiAn+l + M 33 An+l 
Let Ai = Aj^^i - Un. Then, 


pOO 

p(Un) = j p(An+l “ Un, An+l)dAn+i 
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APPENDIX B 


where 


Cl = Mii(4Hi 2 - Hs^) - 4Hi (Mji + Mi 2 f + (M 13 + Ml4f + 4H3 (Mh + M12)(m13 + M 14 ) 

C2 = 2 Mi 3 ( 4 Hi 2 - H32) - 8H1 (Mii + Mi 2 )(Mi 3 + M23) + (mi 3 + Mi 4 )(mi 2 + M22) 

+ 4H3|(Mii + Mi 2 )(Mi 2 + M 22 ) + (M 13 + M23)(Mi 3 + Mi4^ 

2 2 

C3 = M22(4 Hi^ - H3^ j - 4Hi (M13 + M23) + (Mi2 + M22) + 4H3 (Mi 3 + M23 )(Mi 2 + M22) 


From the previous equation for p(Un), can be observed to have a Gaussian proba- 

bility density with a mean of zero and a variance of 



4 |M|(Ci-fC 2 -hC 3 )( 4 Hi 2 -H 32 ) 

4 C 1 C 3 - C 2 ^ 


^1 + ^2 ^3 

4 |m|^ 


The two expressions for the variance resvilt from the fact that the variance appears in 
two places in the equation for the normal probability density fxmction. 
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APPENDIX C 


A RANDOM WALK PROBLEM WITH ABSORBING BOUNDARIES 

In order to find the average number of steps required to reach either the plus or 
minus boundary in the first-order predictor, a random walk problem with absorbing 
boimdaries (ref. 5) is solved. The expected value of Z^, the difference between the 
actual value of the sample and the predicted value, is 

E(Zn) = nE(U) 


if 


n 

Zn= ^ Ui 
i=l 

At the time when the data reach one of the boundaries, Zj^ = -L|2 or = Lu, 
let q equal the probability that Zn = -Lj25 therefore, (1 - q) is the probability that 
Zn g Lii. 

Let be the moment-generating function of Z^ (ref. 6). 

i//z(t) = E(e*^“) = ^ Pje*^^ 

j 

Assume that the length of the step is small compared with the distance to the boundary. 
This assumption, in effect, means that the first sample to occur outside the boundary 
will occim exactly on the boundary and that when the boundary is crossed, Zj^ can have 
one of two values, Ljj or -LJ2- Since by definition (ref. 6), 

e(x2) = ^ p^j^2 

k=l 

E(Zn^) = Li 2Q + Lll[i - q] (Cl) 

Select t = to) so that 

= 1 
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APPENDIX C 


Because of the assumption that Zq can be either Ln or -LJ2 at the boundaries, 

+ (1 - » 1 

(Ljj+Lj2)t„ Li2t„ 

q = — — — 

jLii+Li2)t^, _ ^ 


For t(j, = 0, this equation is indeterminate and must be solved by using L’Hospital's rule. 


„ lim 

*J t -0 
>-0)— u 


(L„ t Li2)efr»*^li^*‘" - Li2e^l2tg, 


(Lii+Li2)t(y 


q = 


(Lll + Ll2)e 


Lii + Li2 - Li 2 Lll 


Lll + ^12 Lll + Li2 


(C2) 


Inserting this expression for q (eq. (C2)) into the equation for E^Z^^) (eq. (Cl)) yields 

E(Zn^) = LllLia 
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*^The aeronautical and space activities of the United States shall he 
conducted so as to contribute ... to the expansion of human knowl- 
edge of phenomena in the atmosphere a)2d space. The Administration 
shall provide for the widest practicable and appropriate dissemination 
of information concerning its activities ajid the results thereof/* 

— National Aeronautics and Space Act of 1958 
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